Encoding a rubric forces you to fully specify it — and a rubric that runs is a rubric you've actually finished
Day 8 of 66
You wrote a structured rubric yesterday. Today you turn it into running code. This isn't a programming exercise dressed up as data work — encoding is the most ruthless specification test there is.
Prose lets you be fuzzy. Code does not. The moment you try to express your rubric as functions and weights, every place you left interpretation latitude becomes a question the computer forces you to answer: What exactly are the weights? At what total does "accept" become "borderline"? What literally happens when the safety gate trips? If you can't write the line, you hadn't actually specified the rule. A rubric you can run is a rubric you've finished.
No Python yet? You're fine — this is the same level as Week 1's taxonomy.py: a dictionary, a function, and one branch. Install Python from python.org, save the file, and run python rubric.py. The thinking is the exercise; Week 7 makes you fluent in the tooling.
The "Try This" box below (injected for build days) contains the full rubric.py. Type it in rather than pasting — you want to read every line, because every line is a decision you made yesterday made explicit. Then replace the sample dimensions and weights with your own real rubric from Day 7.
Read the code for the three decisions it forces you to commit to:
0.45. You can no longer hide behind a word — you've stated exactly how much more.REJECT before the weighted average is even computed. That's the Day 7 rule — gate, don't weight — expressed as control flow. A gate that came after the average would be a bug, and the code makes the ordering impossible to fudge.A final score alone ("3.4, borderline") tells an annotator nothing they can act on. The code's most important line finds and reports the weakest dimension. That single field turns a verdict into direction: "borderline — weakest dimension: technical_quality" tells you exactly why it nearly failed and where the model output needs to improve. A lead designs scoring to emit the reason, not just the number — because the reason is what feeds back into better generations and better guidelines.
This is the difference between a rubric that grades and a rubric that teaches. The same logic generalizes everywhere in this track: good data tooling always surfaces the actionable signal, not just the summary statistic.
Run the code on real outputs. Start with three, then push to ten genuine generations from your own work. As you do, watch for the most valuable event in the whole exercise: the code's verdict disagrees with your gut.
Your gut says "obvious accept," the rubric says "borderline." Do not just override it. Stop and diagnose, because exactly one of two things is true:
Either way you end up with a more specified rubric. A silent override learns nothing; an investigated disagreement upgrades the guideline. This is the Judgment Ritual from Week 1, applied to your own artifact.
Keep a running note of every disagreement and how you resolved it. That note is the raw material for your guideline changelog on Day 9 — and proof to a future employer that you iterate on specifications with evidence.
rubric.py from the Try This box and got it scoring one sampleAn annotator can hold a fuzzy rubric in their head and quietly fill the gaps with judgment. A lead encodes the rubric so the gaps become questions the computer refuses to skip — the exact weights, the threshold where accept becomes borderline, the branch that fires when the gate trips. The altitude jump is from "I apply the spec" to "I prove the spec is finished by making it run, then make it emit the reason a thing failed, not just the score."
Say this in an interview: "I treat code as a lie detector for vague guidelines — if I can't write the line, I never actually specified the rule. And when the code's verdict disagrees with my gut, I don't override it; exactly one of them is underspecified, and investigating which upgrades the rubric every time."